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A.  Period  of  August  1,  2009— July  31,  2010. 

Research  conducted  over  this  period  culminated  in  the  submission  of  five  new  research  papers 
and  revision  of  the  two  submissions  made  over  the  period  2008-2009.  The  latter  have  now  been 
accepted  for  publication.  A  detailed  description  of  the  new  submissions  is  provided  below.  Thesis 
advising  of  two  Ph.D.  students  (Xin  Liu  and  Dominik  Reinhold)  continued  during  this  period.  Two 
new  Ph.D.  students  (Jiang  Chen  and  Xuan  Wang)  began  working  on  their  dissertations.  The  topics 
of  their  study  are  outlined  below.  Invited  presentations  that  were  given  during  this  period  are  also 
listed. 


A. I  Some  Asymptotic  Results  for  Near  Critical  Branching  Processes  [15].  Near  critical 
single  type  Bienayme-Galton- Watson  (BGW)  processes  are  considered.  It  is  shown  that,  under 
appropriate  conditions,  Yaglorn  distributions  of  suitably  scaled  BGW  processes  converge  to  that 
of  the  corresponding  diffusion  approximation.  Convergences  of  stationary  distributions  for  Q- 
processes  and  models  with  immigration  to  the  corresponding  distributions  of  the  associated  diffusion 
approximations  are  established  as  well.  Although  most  of  the  work  is  concerned  with  the  single 
type  case,  similar  results  for  multitype  settings  can  be  obtained.  As  an  illustration,  convergence 
of  Yaglorn  distributions  of  suitably  scaled  multitype  subcritical  BGW  processes  to  that  of  the 
associated  diffusion  model  is  established.  This  work  is  part  of  the  dissertation  work  of  Mr.  Dominik 
Reinhold. 

A. II  Large  Deviation  Properties  of  Weakly  Interacting  Processes  via  Weak  Conver¬ 
gence  Methods  [16].  Collections  of  weakly  interacting  random  processes  have  long  been  of 
interest  in  statistical  physics,  and  more  recently  have  appeared  in  problems  of  engineering  and  op¬ 
erations  research.  A  simple  but  important  example  of  such  a  collection  is  a  group  of  particles,  each 
of  which  evolves  according  to  the  solution  of  an  Ito  type  stochastic  differential  equation  (SDE).  All 
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particles  have  the  same  functional  form  for  the  drift  and  diffusion  coefficients.  The  coefficients  of 
particle  i  are,  as  usual,  allowed  to  depend  on  the  current  state  of  particle  i.  but  also  depend  on 
the  current  empirical  distribution  of  all  particle  locations.  When  the  number  of  particles  is  large 
the  contribution  of  any  given  particle  to  the  empirical  distribution  is  small,  and  in  this  sense  the 
interaction  between  any  two  particles  is  considered  weak.  For  various  reasons,  including  model 
simplification  and  approximation,  one  may  consider  a  functional  law  of  large  numbers  (LLN)  limit 
as  the  number  of  particles  tends  to  infinity.  The  limit  behavior  of  a  single  particle  (under  assump¬ 
tions  which  guarantee  that  all  particles  are  in  some  sense  exchangeable)  can  be  described  by  a  two 
component  Markov  process.  One  component  corresponds  to  the  state  of  a  typical  particle,  while 
the  second  corresponds  to  the  limit  of  the  empirical  measures.  Again  using  that  all  particles  are 
exchangeable,  under  appropriate  conditions  one  can  show  that  the  second  component  coincides 
with  the  distribution  of  the  particle  component.  The  limit  process,  which  typically  has  an  infinite 
dimensional  state,  is  sometimes  referred  to  as  a  nonlinear  diffusion.  Because  the  particle’s  own 
distribution  appears  in  the  state  dynamics,  the  partial  differential  equations  that  characterize  ex¬ 
pected  values  and  densities  associated  with  this  process  are  nonlinear,  and  hence  the  terminology. 
In  this  paper  we  consider  the  large  deviation  properties  of  the  particle  system  as  the  number  of 
particles  tends  to  infinity.  Thus  the  deviations  we  study  are  those  of  the  empirical  measure  of 
the  prelimit  process  from  the  distribution  of  the  nonlinear  diffusion.  In  the  paper  we  develop  an 
approach  which  is  very  different  from  the  one  taken  in  previous  works.  Unlike  classical  techniques, 
our  proofs  do  not  involve  any  time  or  space  discretization  of  the  system  and  no  exponential  prob¬ 
ability  estimates  are  invoked.  The  main  ingredients  in  the  proof  are  weak  convergence  methods 
for  functional  occupation  measures  and  certain  variational  representation  formulas.  Our  proofs 
cover  models  with  degenerate  noise  and  allow  for  interaction  in  both  drift  and  diffusion  terms.  In 
fact  the  techniques  are  applicable  to  a  wide  range  of  model  settings  and  an  example  of  stochastic 
delay  equations  is  considered  to  illustrate  the  possibilities.  The  starting  point  of  our  analysis  is 
a  variational  representation  for  moments  of  nonnegative  functionals  of  a  Brownian  motion.  Using 
this  representation,  the  proof  of  the  large  deviation  principle  reduces  to  the  study  of  asymptotic 
properties  of  certain  controlled  versions  of  the  original  process.  The  key  step  in  the  proof  is  to 
characterize  the  weak  limits  of  the  control  and  controlled  process  as  the  large  deviation  parameter 
tends  to  its  limit  and  under  the  same  scaling  that  applies  to  the  original  process.  More  precisely, 
one  needs  to  characterize  the  limit  of  the  empirical  measure  of  a  large  collection  of  controlled  and 
weakly  interacting  processes.  In  the  absence  of  control  this  characterization  problem  reduces  to  an 
LLN  analysis  of  the  original  particle  system,  which  has  been  studied  extensively.  Our  main  tools 
for  the  study  of  the  controlled  analogue  are  functional  occupation  measure  methods.  Indeed,  these 
methods  have  been  found  to  be  quite  useful  for  the  study  of  averaging  problems,  but  where  the 
average  is  with  respect  to  a  time  variable.  In  the  problem  studied  here  the  measure  valued  processes 
of  interest  are  obtained  using  averaging  over  particles  rather  than  the  time  variable.  Variational 
representations  for  Brownian  motions  and  Poisson  random  measures[9,  10,  12]  have  proved  to  be 
extremely  useful  for  the  study  of  small  noise  large  deviation  problems  and  many  recent  papers 
(see  references  in  [12])  have  applied  these  results  to  a  variety  of  infinite  dimensional  small  noise 
systems.  We  expect  the  current  work  to  be  similarly  a  starting  point  for  the  study,  using  variational 
representations,  of  a  rather  different  collection  of  large  deviation  problems,  namely  asymptotics  of 
a  large  number  of  interacting  particles. 

A. Ill  Multiscale  Diffusion  Approximations  for  Stochastic  Networks  in  Heavy  Traffic 
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[18]-  Jackson  networks  have  been  very  well  studied  and  extensively  used  for  modeling  and  analysis 
in  a  variety  of  disciplines.  The  elegant  distributional  theory  and  asymptotic  properties  of  Jackson 
networks  break  down  when  one  attempts  to  incorporate  some  more  realistic  features  of  specific 
application  settings  into  such  a  model.  For  example,  when  the  distributions  of  the  primitives 
are  relaxed  to  be  more  general  than  i.i.d.  exponential,  or  the  rates  of  interarrivals  and  services  are 
allowed  to  depend  on  the  state  of  the  system,  the  tractability  of  such  a  generalized  Jackson  network 
fails  and  thus  one  seeks  suitable  approximate  models.  One  class  of  such  approximations  are  diffusion 
models  that  can  be  rigorously  justified  when  networks  are  operating  in  the  heavy  traffic  regime, 
i.e.,  when  the  network  capacity  is  roughly  balanced  with  network  load.  Attractiveness  of  such 
approximations  primarily  lies  in  the  fact  that,  analogous  to  the  central  limit  theory,  the  limit  model 
is  described  only  using  a  few  important  parameters  of  the  underlying  networks  and  the  complex 
distributional  properties  of  the  primitives  are  averaged  out.  The  first  general  result  in  the  study  of 
such  diffusion  approximations  is  due  to  Reiman[27]  who  considered  the  case  where  the  arrivals  and 
services  are  mutually  independent  renewal  processes  with  square  integrable  summands.  Queueing 
networks  in  which  rates  of  arrival  and  service  processes  depend  on  the  current  state  of  network 
arise  in  many  application  areas.  In  particular,  Yamada[29]  has  shown  that,  under  appropriate 
heavy  traffic  conditions,  suitably  scaled  queue  length  processes  for  such  networks  converge  weakly 
to  a  reflected  diffusion  process  with  drift  and  diffusion  coefficients  that  depend  on  the  state  of  the 
process. 

In  models  considered  in  works  of  Reiman  and  Yarnada,  the  underlying  topology  of  the  network 
is  the  same  as  that  of  a  Jackson  network.  In  particular,  there  is  a  fixed  probability  routing  matrix  P 
which  governs  the  routing  of  jobs  in  the  networks.  In  this  work,  we  are  interested  in  settings  where 
the  routing  structure  and  arrival/service  rates  may  change  over  time,  according  to  an  extraneous 
finite  state  Markov  process.  This  Markov  process  can  be  interpreted  as  the  random  environment  in 
which  the  system  is  operating.  We  begin  by  considering  a  situation  where  the  system  performance 
is  analyzed  on  time  scales  which  are  much  larger  than  typical  interarrival/service  times  of  jobs, 
which  in  turn  are  significantly  larger  than  typical  time  intervals  at  which  the  state  of  the  random 
environment  changes.  Such  multi-scale  models  are  motivated  by  the  study  of  large  computer 
networks  where  one  is  interested  in  the  behavior  of  modeling  traffic  of  files  with  moderate  size  over 
a  long  period  of  time,  for  a  small  subset  of  nodes  in  the  system.  Denote  by  8  the  collection  of  all 
nodes  in  the  network  and  Sq  C  S  the  subset  of  nodes  of  interest.  Here,  the  size  of  So  is  much  smaller 
than  that  of  8.  One  is  interested  in  building  a  model  for  traffic  between  nodes  in  So  without  taking 
a  very  precise  account  of  the  interactions  of  such  nodes  with  those  in  8  \  So-  A  node  e  €  So  may 
receive  files  from  a  large  number  of  nodes,  most  of  which  one  does  not  want  to  explicitly  account 
for  in  the  reduced  model.  The  point  of  view  taken  here  is  to  model  the  effect  of  nodes  in  8  \  So 
at  a  node  e  G  So  by  a  rapidly  varying  channel  capacity  (at  e),  which  is  modulated  by  a  Markov 
process.  If  a  large  number  of  nodes  in  8  \  8q  are  connected  to  e,  one  expects  that  the  rate  at  which 
the  channel  capacity  changes  is  much  greater  than  the  transmission  rate  of  a  typical  file  through 
e.  Rapid  changes  in  channel  capacity  lead  to  variations  in  processing  rates  and  available  routing 
options  for  files  accepted  at  nodes  in  So.  Thus  we  propose  a  traffic  model  for  nodes  in  So  in  term 
of  a  Jackson  type  network  where  the  arrival/service  rates  and  routing  probability  matrices  vary 
randomly  over  time  according  to  a  finite  state  Markov  process. 

Goal  of  this  work  is  to  study  diffusion  approximations  for  such  multiscale  models.  We  show 
that,  under  appropriate  conditions,  the  state  of  the  system  can  be  well  approximated  by  that  of  a 
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reflected  diffusion  process  with  drift  and  diffusion  coefficients  depend  on  the  equilibrium  measure 
of  the  Markov  process  governing  the  random  environment.  A  crucial  requirement  in  typical  heavy 
traffic  approximation  results  is  the  traffic  balance  condition  of  the  form  A  =  [I  —  P\a.  In  the  multi¬ 
scale  setting  considered  here  all  these  quantities  in  the  above  equation,  i.e.,  P,\,a,  vary  randomly 
over  time  according  to  the  modulating  Markov  process.  We  find  that  if  transition  times  of  the 
Markov  process  are  suitably  fast,  then  the  traffic  balance  condition  can  be  relaxed  to  an  average 
balance  condition  where  the  average  is  taken  with  respect  to  the  equilibrium  measure  of  the  Markov 
process.  Making  this  statement  mathematically  precise  is  one  of  the  key  contributions  of  this  work. 

In  contrast  to  the  scaling  regime  described  above  is  the  situation  where  the  random  environment 
changes  very  slowly  relative  to  the  rates  of  interarrival  and  service  times.  In  such  a  regime,  changes 
in  the  environment  are  revealed  only  when  the  system  is  viewed  over  long  time  intervals.  Our 
second  result  in  this  work  is  to  give  a  precise  mathematical  formulation  and  establish  a  suitable 
diffusion  approximation  for  networks  in  such  a  scaling  regime.  For  simplicity  we  restrict  ourselves 
to  a  setting  where  the  random  environment  affects  only  the  arrival  and  service  rates  in  the  system 
while  the  routing  probabilities  stay  the  same  (governed  by  a  fixed  non-random  matrix  P).  We  show 
that,  with  a  suitable  scaling,  the  state  process  for  the  network  approaches  a  reflected  diffusion  with 
coefficients  that  are  modulated  by  a  Markov  process  that  is  independent  of  the  driving  Brownian 
motion. 

The  two  scaling  regimes  considered  above  correspond  to  very  different  behaviors  of  the  back¬ 
ground  environment  process.  In  the  first  setting  the  environment  changes  at  a  much  faster  rate 
than  the  typical  arrival/serive  rates  in  the  system,  while  in  the  second  one  the  reverse  is  true.  More 
generally,  one  can  consider  a  setting  where  background  variation  of  both  types  exists.  Namely, 
there  are  two  independent  Markov  processes  governing  the  randomly  varying  environment,  the  first 
corresponds  to  fast  changes  while  the  second  captures  slow  changes.  We  also  establish  an  appro¬ 
priate  diffusion  approximation  result  for  such  multi-scale  queueing  networks.  This  work  is  part  of 
the  dissertation  work  of  Ms.  Xin  Liu. 

A. IV  Action  Time  Sharing  Policies  for  Ergodic  Control  of  Markov  Chains  [17].  Markov 
Decision  processes  are  used  extensively  as  the  simplest  models  that  involve  both  stochastic  behavior 
and  control.  A  common  measure  of  performance  is  the  long-time  average  (or  ergodic)  criterion. 
Given  all  relevant  parameters,  a  typical  goal  is  to  find  a  simple  (e.g.  feedback,  or  deterministic 
stationary)  policy  that  achieves  the  optimal  value. 

The  goal  of  adaptive  control  is  to  obtain  an  optimal  policy,  when  some  relevant  information  con¬ 
cerning  the  behavior  of  the  system  is  missing.  The  relevant  information  needs  to  be  obtained  while 
controls  are  chosen  at  each  step.  The  classical  approach  is  to  design  an  algorithm  which  collects 
information,  while  at  the  same  time  choosing  controls,  in  such  a  way  that  sufficient  information  is 
collected  for  making  good  control  decisions,  in  the  sense  that  the  chosen  controls  “approach  opti¬ 
mality  over  time.”  Existing  results  include  general  solutions  for  the  case  of  countable  state  space, 
and  specify  an  estimation  and  a  control  scheme.  A  different  approach  to  this  issue,  including  PAC 
criteria,  can  be  found  in  the  large  literature  on  Reinforcement  learning. 

We  are  concerned  with  a  more  elementary  question,  namely:  What  are  the  basic  controlled 
objects  that  determine  the  cost?  Since  the  objective  function  is  defined  as  a  Cesaro  limit,  we 
can  expect  that  a  similar  Cesaro  definition  of  the  choice  of  controls  would  suffice  to  determine 
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the  cost.  For  the  case  of  countable  state  and  action  spaces  one  fundamental  result  says  that  the 
control  decisions  can  deviate  from  those  dictated  by  the  Markov  policy  q,  and  still  produce  the 
same  long  term  average  cost,  as  long  as  the  conditional  frequencies  converge  to  the  correct  values. 
This  flexibility  is  useful  in  many  estimation  and  sampling  applications.  In  the  current  work  we  are 
concerned  with  a  setting  where  the  state  and  action  spaces  are  not  (necessarily)  countable.  Our 
main  objective  is  to  formulate  an  appropriate  definition  for  an  Action-Time  Sharing  (ATS)  policy 
for  a  given  Markov  control  which,  similar  to  the  countable  case,  on  the  one  hand  leads  to  long  term 
costs  that  are  identical  to  those  for  the  corresponding  Markov  control,  while  on  the  other  hand  allows 
for  flexible  implementation  well  suited  for  various  estimation  and  adaptive  control  goals.  We  show 
that,  under  suitable  stability,  irreducibility  and  Feller  continuity  conditions,  occupation  measures 
for  state  and  action  sequences,  under  an  ATS  policy  (suitably  defined),  converge  a.s.  to  the  same 
(deterministic)  measure  as  under  the  corresponding  Markov  control.  Such  a  result  in  particular 
shows  that  long  term  costs  for  a  broad  family  of  one  stage  cost  functions,  under  the  two  control 
policies,  coincide.  ATS  policies  can  be  used  to  develop  a  variety  of  variance  reduction  schemes  for 
ergodic  control  problems.  Additionally,  ATS  policies  provide  much  flexibility  for  sampling  (namely 
using  controls  without  regards  to  the  ensuing  cost),  for  example  for  the  purpose  of  collecting 
information.  This  could  be  information  which  is  related  to  the  main  optimization  objective,  but 
could  also  be  other  information  which  is  of  interest.  We  show  that  how  ATS  policies  introduced  in 
this  work  can  be  used  for  consistent  estimation  of  unknown  model  parameters  and  also  for  adaptive 
control  problems.  This  paper  is  part  of  the  dissertation  work  of  Ms.  Xin  Liu. 

A.V  Controlled  Stochastic  Networks  in  Heavy  Traffic:  Convergence  of  Value  Functions 
[20].  As  an  approximation  to  control  problems  for  critically- loaded  stochastic  networks,  Harrison 
(1988)  has  formulated  a  stochastic  control  problem  in  which  the  state  process  is  driven  by  a  multi¬ 
dimensional  Brownian  motion  along  with  an  additive  control  that  satisfies  certain  feasibility  and 
non-negativity  constraints.  This  control  problem  that  is  usually  referred  to  as  the  Brownian  Control 
Problem  (BCP)  has  been  one  of  the  key  developments  in  the  heavy  traffic  theory  of  controlled 
stochastic  processing  networks  (SPN).  BCPs  can  be  regarded  as  formal  scaling  limits  for  a  broad 
range  of  scheduling  and  sequencing  control  problems  for  multiclass  queuing  networks.  Finding 
optimal  (or  even  near-optimal)  control  policies  for  such  networks  -  which  may  have  quite  general 
non-Markovian  primitives,  multiple  server  capabilities  and  rather  complex  routing  geometry  -  is 
prohibitive.  In  that  regard  BCPs  that  provide  significantly  more  tractable  approximate  models 
are  very  useful.  In  this  diffusion  approximation  approach  to  policy  synthesis,  one  first  finds  an 
optimal  (or  near-optimal)  control  for  the  BCP  which  is  then  suitably  interpreted  to  construct  a 
scheduling  policy  for  the  underlying  physical  network.  In  recent  years  there  have  been  many  works 
that  consider  specific  network  models  for  which  the  associated  BCP  is  explicitly  solvable  (i.e.  an 
optimal  control  process  can  be  written  as  a  known  function  of  the  driving  Brownian  motions)  and, 
by  suitably  adapting  the  solution  to  the  underlying  network,  construct  control  policies  that  are 
asymptotically  (in  the  heavy  traffic  limit)  optimal.  Although  now  there  are  several  papers  which 
establish  a  rigorous  connection  between  a  network  control  problem  and  its  associated  BCP  by 
exploiting  the  explicit  form  of  the  solution  of  the  latter,  a  systematic  theory  which  justifies  the  use 
of  BCPs  as  approximate  models  has  been  missing.  In  a  recent  work  [21]  it  was  shown  that  for  a 
large  family  of  Unitary  Networks,  with  general  interarrival  and  service  times,  probabilistic  routing, 
and  an  infinite  horizon  discounted  linear  holding  cost,  the  cost  associated  with  any  admissible 
control  policy  for  the  network  is  asymptotically,  in  the  heavy  traffic  limit,  bounded  below  by  the 
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value  function  of  the  BCP.  This  inequality,  which  provides  a  useful  bound  on  the  best  achievable 
asymptotic  performance  for  an  admissible  control  policy  (for  this  large  family  of  models),  was  a  key 
step  in  developing  a  rigorous  general  theory  relating  BCPs  with  SPN  in  heavy  traffic.  The  current 
paper  is  devoted  to  the  proof  of  the  reverse  inequality.  The  network  model  is  required  to  satisfy 
assumptions  made  in  [21].  In  addition  we  impose  a  non-degeneracy  condition,  a  condition  on  the 
underlying  renewal  processes  regarding  probabilities  of  deviations  from  the  mean  and  regularity  of 
a  certain  Skorohod  map.  Under  these  assumptions  we  prove  that  the  value  function  of  the  BCP 
is  bounded  below  by  the  heavy  traffic  limit  (lirnsup)  of  the  value  functions  of  the  network  control 
problem.  Combining  this  with  the  result  obtained  in  [21]  we  obtain  the  main  result  of  the  paper. 
This  theorem  says  that,  under  broad  conditions,  the  value  function  of  the  network  control  problem 
converges  to  that  of  the  BCP.  This  result  provides,  under  general  conditions,  a  rigorous  basis 
for  regarding  BCPs  as  approximate  models  for  critically  loaded  stochastic  networks.  Conditions 
imposed  in  this  paper  allow  for  a  wide  range  of  open  multi-class  queuing  networks,  parallel  server 
networks  and  job  shop  type  models.  We  note  that  our  approach  does  not  require  the  BCP  to 
be  explicitly  solvable  and  the  result  covers  many  settings  where  explicit  solutions  are  unavailable. 
Previous  works  noted  earlier,  that  treat  the  setting  of  explicitly  solvable  BCP,  do  much  more  than 
establish  convergence  of  value  functions.  In  particular,  these  works  give  an  explicit  implementable 
control  policy  for  the  underlying  network  that  is  asymptotically  optimal  in  the  heavy  traffic  limit. 
In  the  generality  treated  in  the  current  work,  giving  explicit  recipes  (eg.  threshold  type  policies)  is 
unfeasible,  however  the  policy  sequence  that  we  construct,  suggests  a  general  approach  for  building 
near  asymptotically  optimal  policies  for  the  network  given  a  near  optimal  control  for  the  BCP. 
Obtaining  near  optimal  controls  for  the  BCP  in  general  requires  numerical  approaches,  study  of 
which  will  be  a  topic  for  future  work. 

A. VI.  Exit  Time  and  Invariant  Measure  Asymptotics  for  Small  Noise  Constrained 
Diffusions  [19].  Diffusions  in  polyhedral  domains  arise  commonly  as  approximate  models  for 
stochastic  processing  networks  in  heavy  traffic.  In  this  work  we  consider  a  family  of  such  constrained 
diffusions  with  a  small  parameter  (denoted  as  e)  multiplying  the  diffusion  coefficient.  Goal  of  the 
work  is  the  study  of  asymptotic  properties  of  invariant  measures  and  exit  times  from  suitable 
domains,  as  e  — ►  0.  The  classical  reference  for  small  noise  asymptotics  of  diffusions  in  is  the 
book  of  Freidlin  and  Wentzell  (1998).  The  basic  object  of  study  in  this  fundamental  body  of  work 
is  a  collection  of  diffusion  processes  {X£}£>q,  given  as 

dX£(t )  =  b(X£(t))dt  +  ea(X£(t))dW(t),  Xe(0)  =  x,  (1) 

where  IF  is  a  A:  dimensional  standard  Brownian  motion  and  b,  a  are  suitable  coefficients.  The 
stochastic  process  X£  =  { X£(t)}o<t<T ,  for  each  T  >  0  can  be  regarded  as  a  Ct  =  C([0,T]  :  Mfc) 
(space  of  continuous  functions  from  [0,  T]  to  with  the  uniform  topology)  valued  random  variable 
and  under  suitable  conditions  on  b,  a  one  can  show  that,  as  e  — >  0,  X£  converges  in  probability  to 
£  which  is  the  unique  solution  of  the  ordinary  differential  equation  (ODE) 

i  =  m,  £(o)  =  z.  (2) 

One  of  the  basic  results  in  the  field  says  that  for  each  T  >  0,  as  e  — >  0,  X£  satisfies  a  large 
deviation  principle  (LDP)  in  Ct,  uniformly  in  the  initial  condition  x  in  any  compact  set  K ,  with  an 
appropriate  rate  function  It  '■  Ct  [0,  oo].  This  result  is  a  starting  point  for  the  study  of  numerous 
asymptotic  questions  for  such  small  noise  diffusions.  In  particular,  when  the  underlying  diffusions 


6 


have  suitable  stability  properties,  the  above  LDP  plays  a  central  role  in  the  study  of  asymptotic 
properties  of  invariant  measures  and  exit  times  from  domains.  The  asymptotics  are  governed  by  the 
“quasi-potential”  function  V  which  is  determined  from  the  collection  of  rate  functions  {It  ■  T  >  0}. 

Freidlin-Wentzell  theory  has  been  extended  and  refined  in  many  different  directions.  One  no¬ 
table  work  is  Day(1982)  which  studies  asymptotics  of  solutions  of  Dirichlet  problems  associated 
with  diffusions  given  by  (1).  To  signify  the  dependence  on  the  initial  condition,  denote  the  solution 
of  (1)  by  X£.  Let  B  be  a  bounded  domain  in  Rfc  and  K  be  an  arbitrary  compact  subset  of  B. 
Under  the  assumption  that  all  solutions  of  the  ODE  (2),  with  x  =  £(0)  E  B,  converge  without 
leaving  B ,  to  a  single  linearly  asymptotically  stable  critical  point,  Day  shows  that  with  suitable 
conditions  on  the  coefficients,  for  all  bounded  measurable  / 

sup 

x,y£K 

converges  to  0  at  an  exponential  rate.  Here,  r|  =  inf{t  :  X£{t)  6  Bc}.  This  property,  usually 
referred  to  as  “exponential  leveling”,  says  that  although  the  exit  time  of  the  process  from  the 
domain  approaches  oo,  as  e  — >  0,  the  moments  of  functionals  of  exit  location,  corresponding  to 
distinct  initial  conditions,  coalesce  asymptotically,  at  an  exponential  rate.  The  key  ingredient  in 
the  proof  is  the  gradient  estimate 


sup  |V«£(x)|  <  ce  1//2,  (3) 

xeK 

where  u£  is  the  solution  of  the  Dirichlet  problem  on  B  associated  with  the  diffusion  (1)  with 
boundary  data  /. 

The  goal  of  the  current  work  is  to  develop  the  Freidlin-Wentzell  small  noise  theory  for  a  family 
of  constrained  diffusions  in  polyhedral  cones.  Let  G  C  Rfc  be  convex  polyhedral  cone  in  Rfc  with 
the  vertex  at  origin  given  as  the  intersection  of  half  spaces  Gj ,  i  =  1,2,  Let  nt  be  the  unit 

vector  associated  with  Gi  via  the  relation 

Gi  =  {x  €  Rfc  :  ( x,rii }  >  0}. 

We  will  denote  the  set  {x  E  dG  :  ( x,rii }  =  0}  by  Fj.  With  each  face  Fj  we  associate  a  unit  vector 
rij  such  that  (di,rii)  >  0.  This  vector  defines  the  direction  of  constraint  associated  with  the  face  Fj,. 
Roughly  speaking  such  a  process  evolves  infinitesimally  as  a  diffusion  in  Rfc  and  is  instantaneously 
pushed  back  using  the  oblique  reflection  direction  dj  upon  reaching  the  face  Fj.  Formally,  such  a 
process,  denoted  once  more  as  X£,  can  be  represented  as  a  solution  of  a  stochastic  integral  equation 
of  the  form 

X*(t)  =  r(x  +  J  b{X£{s))ds  +  e j  a(X£(s))dW(s)^  (t),  (4) 

where  T  is  the  Skorohod  map  taking  trajectories  with  values  in  Rfc  to  those  with  values  in  G. 
consistent  with  the  constraint  vectors  {dj,  i  =  1,  •  •  •  N}.  Under  certain  regularity  assumptions  on 
the  Skorohod  map  and  the  usual  Lipschitz  conditions  on  the  coefficients  b  and  a,  our  first  result 
establishes  a  (locally  uniform)  LDP  for  X£,  in  C([0,  T]  :  G)  for  each  T  >  0.  This  result  is  the 
starting  point  for  all  exit  time  and  invariant  measure  estimates  obtained  in  this  work. 
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Stability  properties  of  constrained  diffusions  in  polyhedral  domains  have  been  studied  in  several 
works.  Let 


C  =  {-]T  a-j/k  :  a*  >  0;  *  £  {1, k}}. 


i=  1 


The  paper  [4]  shows  that  under  regularity  of  the  Skorohod  map,  uniform  non- degeneracy  of  a  and 
Lipschitz  coefficients,  if  for  some  5  >  0 


b(x)  £  C(S)  =  {u  £  C  :  dist(u,  dC)  >  5}  for  all  x  £  G, 


(5) 


then  the  constrained  diffusion  X£  is  positive  recurrent  and  consequently  admits  a  unique  invariant 
probability  measure  fi£.  In  the  current  work  we  study  asymptotic  properties  of  /u,£,  as  e  — >  0. 

We  also  consider  asymptotic  properties  of  exit  times  from  a  bounded  domain  B  C  G  that 
contains  the  origin.  One  important  feature  in  this  analysis  is  that  the  stability  condition  (5),  in 
general,  does  not  ensure  that  the  trajectories  of  the  associated  deterministic  dynamical  system 


C  =  r(x+  /  b(Z(s))ds)  (6) 

Jo 

with  initial  condition  in  B  will  stay  within  the  domain  at  all  times.  However,  a  weaker  stability 
property  holds,  namely,  one  can  find  domains  Bo  C  B  such  that  all  trajectories  of  (6)  starting  in 
Bq  stay  within  B  at  all  times. 

A  significant  part  of  this  work  is  devoted  to  the  proof  of  the  exponential  leveling  property 
for  constrained  diffusions.  We  recall  that  the  key  ingredient  in  the  proof  of  such  a  result  for 
diffusions  in  is  the  gradient  estimate  (3)  for  solutions  of  the  associated  Dirichlet  problem.  For 
diffusions  in  domains  with  corners  and  with  oblique  reflection  fields  that  change  discontinuously, 
there  are  no  regularity  (eg.  C 1  solutions)  results  known  for  the  associated  partial  differential 
equations(PDE).  Our  proof  of  the  exponential  leveling  property  is  purely  probabilistic  and  bypasses 
all  PDE  estimates.  The  main  step  in  the  proof  is  the  construction  of  certain  (uniform  in  e) 
Lyapunov  functions  which  are  then  used  to  construct  a  coupling  of  the  processes  X£,  X£  with 
explicit  uniform  estimates  on  exponential  moments  of  time  to  coupling.  The  key  ingredient  in  this 
coupling  construction  is  a  minorization  condition  on  transition  densities  of  reflected  diffusions.  We 
give  some  examples  where  such  a  minorization  property  holds.  Obtaining  general  conditions  under 
which  this  estimate  on  transition  densities  of  reflected  diffusions  holds  is  a  challenging  open  problem. 
The  minorization  property  allows  for  the  construction  of  a  pseudo-atom,  for  each  fixed  £  >  0,  using 
split  chain  ideas  of  Athreya-Ney-Nummelin.  Coupling  based  on  pseudo-atom  constructions  have 
been  used  by  many  authors  for  the  study  of  a  broad  range  of  stability  and  control  problems,  however 
the  current  paper  appears  to  be  the  first  to  bring  these  powerful  techniques  to  bear  for  the  study 
of  exit  time  asymptotics  of  small  noise  Markov  processes. 

As  a  second  consequence  of  our  coupling  constructions  we  show  that  difference  of  moments  of 
an  exit  time  functional  with  a  sub-logarithmic  growth  corresponding  to  distinct  initial  conditions  in 
Bq  is  asymptotically  bounded.  Note  that  for  typical  unbounded  functionals  the  associated  moments 
will  approach  oo,  as  £  — ►  0  thus  the  result  provides  a  rather  non-trivial  “leveling  estimate”.  The 
third  important  consequence  of  our  approach  says  that  as  initial  conditions  approach  0  at  rate  £2 
the  corresponding  moments  of  exit  time  functionals  with  sub-logarithmic  growth  asymptotically 


coalesce  at  an  exponential  rate.  To  the  best  of  our  knowledge,  analogous  estimates  are  not  known 
even  for  the  setting  of  small  noise  diffusions  (with  suitable  stability  properties)  in  Rfc. 

These  estimates  are  interesting  for  one  dimensional  models  as  well.  Let  X  be  a  one  dimensional 
reflected  Brownian  motion,  starting  from  x  £  [0,1),  with  drift  b  £  (— oo,0)  and  variance  cr2e2  £ 
(0,  oo),  given  on  some  probability  space  (D,  X,  P%)-  I.e.,  P%  a.e., 

X(t)  =  x  +  bt  +  eaB(t )  —  inf  {(x  +  bs  +  saB(s))  A  0},  t  >  0 

0<s<t 

where  B  is  a  standard  Brownian  motion.  Let  r  =  inf{f  :  X(t)  =  1}.  Let  :  R+  — >  R+  be  a 
measurable  map  and  let  <^£(x)  =  Wx^{t).  Our  results  for  this  one  dimensional  setting  says  that  if 
if  has  a  sub-logarithmic  growth  then  for  some  c,  5  £  (0,  oo),  as  e  — >  0, 

eS/£\<Pe(ce2)  -  <pe(0)|  — >  0.  (7) 

If  ip(t)  =  t,  t  >  0,  then  ipe  is  the  unique  solution  to  the  boundary  value  problem 

£2ctV"  +  We  =  -!>  ^(0)  =  M1)  =  0, 

whose  solution  is  given  as  <^£(x)  =  ^-[ee2<£  —  ePX1]  +  ^(1  —  x).  It  is  easily  verified  that  if  x£  =  ce  , 
for  some  c  >  0,  then  | (pe(xe)  —  </?e(0)|  — >  0  at  rate  e2,  as  e  — >  0.  In  particular,  the  decay  is  not 
exponential.  Furthermore,  for  x  /  y  in  [0,1),  |<^£(x)  —  <^e(y)|  — >  oo  as  e  — >  0.  For  general  V7 
a  similar  ODE  analysis  is  less  tractable  and  thus  the  coupling  techniques  developed  here  give  a 
powerful  approach  (especially  in  higher  dimensions)  to  the  study  of  such  asymptotic  estimates. 

A. VIII.  Dissertation  Advising.  Four  Ph.D.  students  who  were  partially  supported  by  the  grant 
are  currently  working  under  the  direction  of  the  PI.  Ms.  Xin  Liu  (Topics  in  Diffusion  Approxima¬ 
tions  for  Multiscale  queuing  networks),  Mr.  Dominik  Reinhold  (Long  time  properties  of  Critical 
Branching  Processes),  Ms.  Jiang  Chen  (Large  Deviations  for  Stochastic  Dynamical  Systems)  and 
Mr.  Xuan  Wang  (Scaling  limits  of  Random  Graph  Models). 

A.  IX.  Invited  Presentations. 

•  A  Stochastic  Differential  Game  for  the  Inhomogeneous  Infinity-Laplace  Equation,  Institute  of 
Mathematical  Statistics,  73rd  Annual  Meeting,  Aug  9-13,  2010,  Gothenburg,  Sweden. 

•  Some  Large  Deviation  Problems  for  Infinite  Dimensional  Stochastic  Dynamical  Systems ,  The¬ 
ory  and  Qualitative  Behavior  of  Stochastic  Dynamics,  SAMSI,  February  8-10,  2010. 

•  Elliott- Kalton  stochastic  differential  games  associated  with  the  infinity  Laplacian,  Probability 
Seminars  at  University  of  Nice,  Sophia-Antipolis  July  2010,  Department  of  Mathematics- 
National  Taiwan  University,  May  2010. 

•  Variational  Representations  and  Large  Deviations,  Institute  of  Math.,  Academia  Sinica,  Tai¬ 
wan,  May  2010. 

B.  Period  of  August  1,  2008— July  31,  2009. 
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Research  conducted  over  this  period  culminated  in  the  submission  of  two  new  research  papers 
and  revision  of  four  of  the  five  submissions  made  over  the  period  2007-2008.  Three  of  the  latter 
group  have  now  appeared  or  have  been  accepted  for  publication.  A  detailed  description  of  the 
new  submissions  is  provided  below.  Thesis  advising  of  two  Ph.D.  students  (Xin  Liu  and  Dominik 
Reinhold)  was  carried  out  during  this  period. 


B.I.  On  Near  Optimal  Trajectories  of  Games  Associated  with  the  infinity-Laplacian 

[5] .  Consider  the  equation 


— 2Aocu  =  h  in  G, 

< 

u  =  g  on  dG , 


(8) 


where,  for  an  integer  m  >  2,  G  C  Mm  is  a  bounded  C2  domain,  and  g  £  C(9G,M)  and  the  functions 
h  £  C(G,R  \  {0})  are  given.  The  infinity-Laplacian  is  defined  as 


Aoo  / 


£i 


V  Dj  f  Dijf  Djf 


Df 

W\ 


D2  f 


Df 

\Dff 


provided  Df  f  0,  where  for  a  C 2  function  /  we  denote  by  Df  the  gradient  and  by  D2 f  the  Hessian 
matrix.  This  paper  is  motivated  by  recent  work  of  Peres  et.  al.  [26],  where  a  discrete  time  random 
turn  game,  referred  to  as  Tug-of-War,  is  developed  in  relation  to  (8).  This  game,  parameterized 
by  e  >  0,  has  the  property  that  the  vanishing-e  limit  of  the  value  function  uniquely  solves  (8)  in 
the  viscosity  sense.  The  stochastic  differential  equation  (SDE) 


dXt  =  2  p(Xt)dWt  +  2  q(Xt)dt, 


(9) 


where 

(10) 

is  suggested  in  [26]  as  the  game’s  dynamics  in  the  vanishing-e  limit.  The  relation  is  rigorously 
established  in  examples,  but  only  heuristically  justified  in  general.  Recently,  in  [3],  a  two-player 
zero-sum  stochastic  differential  game  (SDG)  is  considered,  for  which  the  value  function  uniquely 
solves  (8)  in  the  viscosity  sense.  The  dynamics  of  the  state  process  are  given  as 

Xt  =  x  +  f  (As  —  Bs)dWs  +  f  ( Cs  +  DS)(AS  +  Bs)ds ,  t  £  [0, oo),  (11) 

Jo  Jo 

and  (A,  C)  and  ( B,D )  are  control  processes,  chosen  by  the  two  players,  taking  values  in  5m_1  x 
[0,oo)  where  <STO_1  is  the  unit  sphere  in  Mm.  Defined  in  the  Elliott-Kalton  sense,  the  SDG  of  [3] 
is  formulated  in  such  a  way  that  one  of  the  players  selects  a  strategy,  and  then  the  other  selects  a 
control  process.  The  payoff  functional  of  interest  is  an  exit  time  cost  of  the  form  E[fY  h(Xs)ds  + 
g(XT)\,  where  r  =  inf{f  :  Xt  G}  (with  an  appropriate  convention  regarding  r  =  oo).  Some 
comments  on  how  such  a  game  arises  from  a  discrete  Tug  of  War  formulation  can  be  found  in 
Section  B.I. 
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The  goal  of  this  paper  is  to  show  that,  with  appropriate  conditions,  (9)  can  be  rigorously 
interpreted  as  the  optimal  dynamics  of  the  SDG.  We  will  assume  in  this  paper  that  the  equation 
possesses  a  classical  solution  u  i.e.,  C 2  with  non-vanishing  gradient.  Under  this  assumption  we 
specify,  for  each  5  >  0,  a  d-optimal  strategy  /3s,  and  a  control  process  Ys  that  is  ^-optimal  for  play 
against  /3s,  in  terms  of  first  and  second  derivatives  of  u.  We  then  identify  the  limit  law,  as  6  — >  0, 
of  the  state  process  under  ((3s,  Y5),  as  the  solution  X  to  the  SDE  (9),  stopped  when  X  hits  the 
boundary  8G. 

The  construction  of  near  optimal  strategy-control  pairs,  that  is  of  interest  in  its  own  right,  is 
based  on  an  interpretation  of  (8)  as  the  following  Bellman-Isaacs  type  equation 

sup  inf  {  —  o(a  —  b)' (D2u)(a  —  b)  —  (c  +  d) (a  +  b)  ■  Du\  =  h. 

|6|=i,d>o  M=bc>o  f  2  ) 

In  this  form  there  is  a  natural  way  to  construct  strategy  and  control,  by  associating  the  supremum 
and  infimum  with  the  two  players.  The  variables  a,  b,  c  and  d  selected  by  the  players  dictate  the 
coefficients  of  the  game’s  state  process,  and,  we  prove,  the  coefficients  converge  to  those  of  equation 
(9)  in  the  limit  as  the  supremum  and  infimum  are  achieved.  This  convergence  is  then  lifted  to  the 
convergence  of  the  underlying  state  process  in  (11)  to  the  diffusion  (9). 


B.II.  Variational  Representations  for  Continuous  Time  Processes  [12].  In  this  paper  we 
prove  a  variational  representation  for  positive  measurable  functionals  of  a  Poisson  random  measure 
and  an  infinite  dimensional  Brownian  motion.  These  processes  provide  the  driving  noises  for  a 
wide  range  of  important  process  models  in  continuous  time,  and  thus  we  also  obtain  variational 
representations  for  these  processes  when  a  strong  solution  exists.  The  representations  have  a  number 
of  uses,  the  most  important  being  to  prove  large  deviation  estimates. 

The  theory  of  large  deviations  is  by  now  well  understood  in  many  settings,  but  there  remain 
some  situations  where  the  topic  is  not  as  well  developed.  These  are  often  settings  where  technical 
issues  challenge  standard  approaches,  and  the  problem  of  finding  nearly  optimal  or  even  reasonably 
weak  sufficient  conditions  is  hindered  as  much  by  technique  of  proof  as  any  other  issue. 

Variational  representations  of  the  sort  developed  in  this  paper  have  been  shown  to  be  partic¬ 
ularly  useful,  when  combined  with  weak  convergence  methods,  for  analyzing  such  systems.  For 
example,  Brownian  motion  representations  have  been  used  by  many  authors  (see  references  in  [12]) 
in  the  large  deviation  analysis  of  solutions  to  SPDEs  in  the  small  noise  limit.  The  usefulness  of  the 
representations  is  in  part  due  to  the  fact  that  they  avoid  certain  discretization  and/or  approxima¬ 
tion  arguments,  which  can  be  cumbersome  for  complex  systems.  Another  reason  is  that  exponential 
tightness,  a  property  that  is  often  required  by  other  approaches  and  which  often  leads  to  artifi¬ 
cial  conditions,  is  replaced  by  ordinary  tightness  for  controlled  processes  with  uniformly  bounded 
control  costs.  (Although  exponential  tightness  can  a  posteriori  be  obtained  as  consequence  of  the 
large  deviation  principle  (LDP)  and  properties  of  the  rate  function).  What  is  required  for  the  weak 
convergence  approach,  beyond  the  variational  representations,  is  that  basic  qualitative  properties 
(existence,  uniqueness  and  law  of  large  number  limits)  can  be  demonstrated  for  certain  controlled 
versions  of  the  original  process. 
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Previous  work  on  variational  representations  has  focused  on  either  discrete  time  processes, 
functionals  of  finite  dimensional  Brownian  motion,  or  various  formulations  of  infinite  dimensional 
Brownian  motion.  An  important  class  of  processes  that  were  not  covered  are  continuous  time 
Markov  processes  with  jumps,  e.g.,  Levy  processes.  In  this  paper  we  eliminate  this  gap,  and  in 
fact  give  variational  representations  for  functionals  of  a  fairly  general  Poisson  random  measure 
(PRM)  plus  an  independent  infinite  dimensional  Brownian  motion  (BM),  thereby  covering  many 
continuous  time  models. 

In  [30]  Zhang  has  also  proved  a  variational  representation  for  functionals  of  a  PRM.  The  repre¬ 
sentation  in  [30]  is  given  in  terms  of  certain  predictable  transformations  on  the  canonical  Poisson 
space.  Existence  of  such  transformations  relies  on  solvability  of  certain  nonlinear  partial  differen¬ 
tial  equations  from  the  theory  of  mass  transportation.  This  imposes  restrictive  conditions  on  the 
intensity  measure  (e.g.,  absolute  continuity  with  respect  to  Lebesgue  measure)  of  the  PRM,  and 
even  the  very  elementary  setting  of  a  standard  Poisson  process  is  not  covered.  Additionally,  use  of 
such  a  representation  for  proving  large  deviation  results  for  general  continuous  time  models  with 
jumps  appears  to  be  unclear. 

In  contrast,  we  impose  very  mild  assumptions  on  the  intensity  measure  (namely,  it  is  a  a- finite 
measure  on  a  locally  compact  space),  and  establish  a  representation,  that  is  given  in  terms  of  a  fixed 
PRM  defined  on  an  augmented  space.  A  key  question  in  formulating  the  representation  for  PRM 
is  “ what  form  of  controlled  PRM  is  natural  for  purposes  of  representation ?”  In  the  Brownian  case 
there  is  little  room  for  discussion,  since  control  by  shifting  the  mean  is  obviously  very  appealing.  In 
[30]  the  control  moves  the  atoms  of  the  Poisson  random  measure  through  a  rather  complex  nonlinear 
transformation.  The  fact  that  atoms  are  neither  created  nor  destroyed  is  partly  responsible  for  the 
fact  that  the  representation  does  not  cover  the  standard  Poisson  process.  In  the  representation 
obtained  in  our  work  the  control  process  enters  as  a  censoring/ thinning  function  in  a  very  concrete 
fashion,  which  in  turn  allows  for  elementary  weak  convergence  arguments  in  proofs  of  large  deviation 
results. 

As  an  application  of  the  representation,  we  establish  a  general  large  deviation  principle  (LDP) 
for  functionals  of  a  PRM  and  an  infinite  dimensional  BM.  A  similar  LDP  for  functionals  of  an 
infinite  dimensional  BM  [9]  has  been  used  in  recent  years  by  numerous  authors  to  study  small 
noise  asymptotics  for  a  variety  of  infinite  dimensional  stochastic  dynamical  models  (see  references 
in  [12]).  The  LDP  obtained  in  the  current  paper  is  expected  to  be  similarly  useful  in  the  study  of 
infinite  dimensional  stochastic  models  with  jumps  (e.g.,  SPDEs  with  jumps).  We  illustrate  the  use 
of  the  LDP  in  Section  4  via  a  simple  finite  dimensional  jump-diffusion  model.  The  goal  is  to  simply 
show  how  the  approach  can  be  used  and  no  attempt  is  made  to  obtain  the  best  possible  conditions. 

B.III.  Invited  Presentations. 

•  Variational  Representations  and  Large  Deviations,  2009  Barrett  Lectures  at  The  University 
of  Tennessee,  April  17  -  18,  2009. 

•  Variational  Representations  and  Large  Deviations,  33rd  Conference  on  Stochastic  Processes 
and  Their  Applications,  Berlin,  27-31  July,  2009. 

•  Elliott- Kalton  stochastic  differential  games  associated  with  the  infinity  Laplacian,  Probability 
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Seminars  at  Stanford  University,  Univ.  of  Cal.  at  Berkeley,  Columbia  University,  Carnegie 
Mellon  University,  Princeton  University. 

C.  Period  of  July  1,  2007— July  31,  2008. 

Research  conducted  over  the  period  July  1,  2007  -  July  31,  2008  culminated  in  five  research  papers 
that  were  submitted  for  publication.  Four  of  the  five  have  been  accepted  for  publication  while  the 
fifth  is  currently  under  review.  A  detailed  description  of  these  works  is  given  below.  This  period 
also  included  graduation  of  two  Ph.D.  students  supported  by  the  contract  whose  dissertation  results 
are  described  briefly.  Invited  presentations  that  were  given  during  this  period  are  also  listed. 


C.I.  A  stochastic  differential  game  for  the  inhomogeneous  oo-Laplace  equation  [3], 

For  an  integer  rn  >  2  let  a  bounded  C2  domain  G  C  Mm,  and  functions  g  £  C(<9G,M)  and  h  £ 
C(G,  M\  {0})  be  given.  We  study  a  two-player  zero-sum  stochastic  differential  game  (SDG),  defined 
in  terms  of  an  m-dimensional  state  process  that  is  driven  by  a  one-dimensional  Brownian  motion, 
played  until  the  state  exits  the  domain.  The  functions  g  and  h  serve  as  terminal,  and,  respectively, 
running  payoffs.  The  players’  controls  enter  in  a  diffusion  coefficient  and  in  an  unbounded  drift 
coefficient  of  the  state  process.  The  dynamics  are  degenerate  in  that  it  is  possible  for  the  players 
to  completely  switch  off  the  Brownian  motion.  We  show  that  the  game  has  value,  and  characterize 
the  value  function  as  the  unique  viscosity  solution  u  (uniqueness  of  solutions  is  known  from  [26]) 
of  the  equation 


—2/^00^  =  h  in  G, 

< 

=  g  on  dG. 


(12) 


Here  Aa 0  is  the  infinity-Laplacian  defined  as  A^f  =  (D f)' (D2 f)(D f) /\D f\2 ,  provided  Df  /  0, 
where  for  a  C 2  function  /  we  denote  by  Df  the  gradient  and  by  D2 f  the  Hessian  matrix.  Our 
work  is  motivated  by  a  representation  for  u  of  Peres  et.  al.  [26],  as  the  limit,  as  e  — »  0,  of  the 
value  function  V£  of  a  discrete  time  random  turn  game,  referred  to  as  Tug-of-War ,  in  which  e  is  a 
parameter.  The  contribution  of  the  current  work  is  the  identification  of  a  game  for  which  the  value 
function  is  precisely  equal  to  u. 


The  Tug-of-War  game  introduced  in  [26]  is  as  follows.  Fix  e  >  0.  Let  a  token  be  placed  at 
x  £  G,  and  set  Xq  =  x.  At  the  k- th  step  of  the  game  (k  >  1),  an  independent  toss  of  a  fair  coin 
determines  which  player  takes  the  turn.  The  selected  player  is  allowed  to  move  the  token  from  its 
current  position  X^-i  £  G  to  a  new  position  A/,,  in  G,  in  such  a  way  that  \X^  —  Xfc_i|  <  e.  The 
game  ends  at  the  first  time  K  when  Xk  £  dG.  The  associated  payoff  is  given  by 


E 


2  K-l 

g{XK)  +  J  E  h ™ 

k= 0 


(13) 


Player  I  attempts  to  maximize  the  payoff  and  Player  IPs  goal  is  to  minimize  it.  It  is  shown  in  [26] 
that  the  value  of  the  game,  defined  in  a  standard  way  and  denoted  Ue(x),  exists,  that  V£  converges 
uniformly  to  a  function  V  referred  to  as  the  “continuum  value  function” ,  and  that  V  is  the  unique 
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viscosity  solution  of  (12)  (these  results  are  in  fact  also  proved  for  the  homogeneous  case,  and  in 
generality  greater  than  the  scope  of  the  current  paper) .  The  question  of  associating  a  game  directly 
with  the  continuum  value  was  posed  and  some  basic  technical  challenges  associated  with  it  were 
discussed  in  [26]. 


Our  approach  to  the  question  above  is  via  a  SDG  formulation.  To  motivate  the  form  of  the 
SDG  we  start  with  the  Tug-of-War  game  and  present  some  formal  calculations.  Let  {£&,  k  £  N}  be 
a  sequence  of  i.i.d.  random  variables  on  some  probability  space  (fl,  P,  P )  with  P(^k  =  1)  =  P(£k  = 
—  1)  =  1/2,  interpreted  as  the  sequence  of  coin  tosses.  Let  {Pk}k>o  be  a  filtration  of  P  to  which  {£*,} 
is  adapted  and  such  that  {£fc+i,£fc+2  . . .}  is  independent  of  Pi-  for  every  k  >  0.  Let  {a.^} ,  {frfc}  be 
{■Ffcj-predictable  sequences  of  random  variables  with  values  in  Be(0)  =  {x  £  Rm  :  |x|  <  e}.  These 
sequences  correspond  to  control  actions  of  Player  I  and  II;  that  is,  ak  [resp.,  bk\  is  the  displacement 
exercised  by  Player  I  [resp.,  Player  II]  if  it  wins  the  fc-th  coin  toss.  Associating  the  event  {£&  =  1} 
with  Player  I  winning  the  A:-th  toss,  one  can  write  the  following  representation  for  the  position  of 
the  token,  starting  from  initial  state  x.  For  j  £  N, 


Xj  —  x  T 


£[■ 

k=\ 


1  +  fk  ,  u  1  -  6: 
ak  — X - 1-  bk  — - — 


j  j 

Eflfc  ~  bk  \r-^  ak  +  bk 

o  sfc  +  9 


k= 1 


k=\ 


We  shall  refer  to  {Xj}  as  the  ‘state  process’.  This  representation,  in  which  turns  are  not  taken 
at  random  but  both  players  select  an  action  at  each  step,  and  the  noise  enters  in  the  dynamics,  is 
more  convenient  for  the  development  that  follows.  Let  e  =  1  / \fn  and  rescale  the  control  processes 
by  defining,  for  t  >  0,  A/  =  /nay,  =  yj nb\nty  Consider  the  continuous  time  state  process 
XI1  =  X\nt-\,  and  define  {W”}i> o  by  setting  Wq  =  0  and  using  the  relation 


W?  =  W(k-l)/n  +  H - 


k-  1 


n 


Vnt,k, 


t  £ 


k-  1 


n 


k  £  N. 


Then  we  have 

xt  =  x  +  \  J\a ns  -  Bg  ) dWg  +\  j^  Vn(A ?  +  B?)ds.  (14) 

Note  that  Wn  converges  weakly  to  a  standard  Brownian  motion,  and  since  |A/|  V  \B^\  <  1,  the 
second  term  on  the  right  hand  side  of  (14)  forms  a  tight  sequence.  Thus  it  is  easy  to  guess  a 
substitute  for  it  in  the  continuous  game.  Interpretation  of  the  asymptotics  of  the  third  term  is 
more  subtle,  and  is  a  key  element  of  the  formulation.  One  possible  approach  is  to  replace  the 
factor  y/n  by  a  large  quantity  that  is  dynamically  controlled  by  the  two  players.  This  point  of  view 
motivates  one  to  consider  the  identity 


-2/1^/=  sup  inf  -  b)'{D2f)  (a  -  b)  -  (c  +  d){a  +  b)  ■  Df\  , 

|6|=1, d>0  M=bc>0  [2  J  (15) 

feC2,  Df  +  0, 

for  the  following  reason.  Let  ki  =  5m_1  x  [0,oo)  where  Sm~l  is  the  unit  sphere  in  Rm.  The 
expression  in  curly  brackets  is  equal  to  Ca,b,c,d f{x),  where  for  ( a,c),(b,d )  £  Tt,  Ca)b,C)d  is  the 
controlled  generator  associated  with  the  process 

Xt  =  x  +  f  ( As  —  Bs)dWs  +  f  ( Cs  +  DS)(AS  +  Bs)ds ,  t  £  [0, 00),  (16) 

Jo  Jo 
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and  (A,  C )  and  (B,  D )  are  control  processes  taking  values  in  TL.  Since  A0 0  is  related  to  (14)  via  the 
Tug-of-War,  and  Ca,b,c,d  to  (16),  identity  (15)  suggests  to  regard  (16)  as  a  formal  limit  of  (14).  Thus 
the  SDG  we  consider  has  (16)  as  a  state  process,  where  the  controls  {A,  C )  and  ( B ,  D)  are  chosen 
by  the  two  playes.  Finally,  the  payoff  functional,  as  a  formal  limit  of  (13),  and  accounting  for  the 
extra  factor  of  1/2  in  (14),  is  given  as  E[f0  h(Xs)ds  +  g(XT)\,  where  r  =  inf{f  :  Xt  (/  G}  (with  an 
appropriate  convention  regarding  r  =  oo).  We  show  that  the  zero  sum  SDG,  where  player  I  chooses 
(A,  C )  and  seeks  to  maximize  the  payoff  whereas  player  II  chooses  ( B ,  D)  aiming  to  minimize  the 
payoff,  has  a  value.  Furthermore  the  value  function  is  the  unique  viscosity  solution  of  (12).  Thus 
our  result  provides  a  game  theoretic  interpretation  of  the  continuum  value  of  [26]  and  also  gives  a 
stochastic  representation  for  the  solution  of  the  nonlinear  degenerate  elliptic  PDE  (12). 


C.II.  Ergodic  Rate  Control  Problem  for  Single  Class  Queueing  Networks  [14].  We  study 
Jackson  networks  with  state  dependent  and  dynamically  controlled  arrival  and  service  rates.  The 
network  consists  of  K  service  stations,  each  of  which  has  an  associated  infinite  capacity  buffer. 
Arrivals  of  jobs  can  be  from  outside  the  system  and/or  from  internal  routing.  Upon  completion  of 
service  at  a  station,  the  customer  is  routed  to  one  of  the  other  service  stations  (or  exits  the  system) 
according  to  a  probabilistic  routing  matrix.  Network  is  assumed  to  be  in  heavy  traffic  in  an 
appropriate  sense.  Roughly  speaking,  one  considers  a  sequence  of  networks  with  identical  topology, 
parametrized  by  n  G  N.  Instantaneous  arrival  and  service  rates  in  the  n-th  network  are  queue 
length  dependent  and  are  of  order  0(n).  Additionally,  a  system  manager  can  exercise  service  rate 
controls  of  order  0(n1//2).  The  heavy  traffic  assumption  says  that  traffic  intensity  at  each  station 
is  of  the  order  1  —  0(l/n1//2).  In  an  uncontrolled  setting,  diffusion  approximations  of  suitably 
scaled  queue  length  processes  for  such  networks  have  been  studied  in  [29].  The  limit  stochastic 
process  is  a  reflected  diffusion  in  the  nonnegative  orthant.  Dependence  of  arrival  and  service  rates 
on  queue  length  processes  leads  to  drift  and  diffusion  coefficients  for  the  limit  model  that  are  state 
dependent.  Existence  and  uniqueness  of  solutions  to  such  reflected  stochastic  differential  equations 
(SDE)  follows  from  the  classical  theory  and  well  understood  regularity  properties  of  the  Skorohod 
map. 

For  the  controlled  setting  considered  here,  a  formal  analysis  along  similar  lines  leads  to  controlled 
reflected  diffusions  with  control  entering  linearly  in  the  drift  coefficient.  Goal  of  this  work  is  to 
use  such  a  formal  diffusion  approximation  in  order  to  obtain  provably  asymptotically  optimal  rate 
control  policies  for  the  underlying  physical  networks.  We  are  concerned  with  an  average  long  term 
cost  per  unit  time  criterion.  Cost  function  is  a  sum  of  two  terms;  the  first  term  measures  the 
cost  for  holding  customers  in  the  buffer,  while  the  second  term  is  the  cost  for  exercising  control. 
The  holding  cost  is  allowed  to  have  a  polynomial  growth  (as  a  function  of  queue  lengths)  while 
the  control  cost  is  assumed  to  be  linear.  Since  the  cost  criterion  involves  an  infinite  time  horizon, 
additional  stability  conditions  are  needed  to  ensure  the  finiteness  of  the  cost.  In  our  work  these 
conditions  are  manifested  in  the  definition  of  the  class  of  admissible  controls.  The  definition  in 
particular  ensures  that  the  controls  are  “uniformly  stabilizing”,  viz.  all  polynomial  moments  of 
the  queue  length  process  are  bounded  uniformly  in  time,  the  scaling  parameter  and  the  choice  of 
admissible  control  sequence.  This  stability  property  is  a  key  ingredient  in  our  proofs. 

Our  main  result  relates  the  value  function  (i.e.  the  optimum  value  of  the  cost)  of  the  queueing 
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rate  control  problem  with  the  value  function  associated  with  an  ergodic  cost  problem  for  the  formal 
controlled  diffusion  limit.  Cost  function  for  the  diffusion  model  is  analogous  to  that  for  the  physical 
networks.  It  is  shown  that,  under  certain  conditions,  the  value  function  for  the  n-th  queueing 
network  converges  to  that  for  the  diffusion  model,  as  n  — >  oo.  The  theorem  also  shows  that  using 
an  appropriately  chosen  e-optimal  feedback  control  for  the  diffusion  model,  one  can  construct  an 
asymptotically  e-optimal  rate  control  policy  for  the  physical  queueing  network.  Thus  the  theorem 
describes  the  precise  sense  in  which  the  diffusion  model,  which  is  originally  justified  only  formally, 
is  a  rigorous  asymptotic  approximation  for  the  queueing  model.  Rate  control  problems  with  an 
ergodic  cost  criterion  for  single  class  open  queueing  networks  in  heavy  traffic  have  been  studied  in 
several  papers.  Most  works  concern  a  one  dimensional  problem  (i.e.  a  single  server  queue)  and 
provide  explicit  expressions/recipes  for  asymptotically  optimal  policies.  The  paper  [25]  studies  a 
general  K  dimensional  Jackson  type  network  with  finite  buffers.  In  this  setting,  due  to  compactness 
of  the  state  space,  all  stability  issues  become  trivial.  Furthermore,  convergence  of  value  functions 
is  established  under  the  additional  assumption  that  e-optimal  continuous  feedback  controls  exist 
for  the  limit  diffusion  model-  but  no  details  on  when  such  an  assumption  holds  are  provided. 

One  of  the  main  steps  in  the  proof  of  our  main  result  is  establishing  existence  of  an  e-optimal 
continuous  feedback  control  for  the  limit  diffusion  model.  Existence  of  optimal  feedback  controls 
for  the  class  of  ergodic  diffusion  control  problems  studied  here  has  been  established  in  [7].  Our 
proof  starts  from  such  an  optimal  control  b*  and  constructs  a  sequence  {bn}  of  continuous  feedback 
controls  in  a  manner  such  that  an  appropriately  chosen  measure  of  the  set  {bn  ^  b*}  converges  to  0 
as  n  — ■>  oo.  We  show  that,  as  long  as  initial  distributions  converge,  the  solution  Xn  of  the  reflected 
SDE  that  uses  bn  as  the  feedback  control  converges  weakly  to  X  which  solves  the  equation  with  bn 
replaced  with  b*.  Proof  of  this  result  relies  on  certain  representations  and  estimates  for  transition 
probability  densities  of  reflected  Brownian  motions  that  are  of  independent  interest.  Once  weak 
convergence  of  Xn  to  X  is  established,  existence  of  an  e-optimal  continuous  feedback  control  is  an 
immediate  consequence  of  the  linearity  of  the  control  cost.  Next,  an  e-optimal  continuous  feedback 
control  b£ ,  obtained  from  above  result  is  used  to  define  a  rate  control  policy  for  the  n-th  network, 
for  each  n  E  N.  The  associated  costs  are  shown  to  converge  to  the  cost  associated  with  bs  in  the 
diffusion  control  problem.  As  a  consequence  of  this  result  one  obtains  that  the  value  function  for 
the  limit  diffusion  model  is  an  upper  bound  for  any  limit  point  of  the  sequence  of  value  functions 
for  the  controlled  queueing  networks.  We  then  establish  the  reverse  inequality.  Combining  these 
bounds  we  have  the  main  result  of  the  paper.  Proofs  of  these  bounds  use  functional  occupation 
measure  methods  developed  in  [25] .  The  main  idea  is  to  represent  the  cost  associated  with  the  n-th 
network  in  terms  of  integrals  with  respect  to  occupation  measures  associated  with  various  stochastic 
processes  that  constitute  the  dynamical  description  of  the  network.  Using  stability  estimates  of 
this  paper  one  can  establish  tightness  of  these  occupation  measures  and  then  classical  martingale 
characterization  methods  can  be  applied  to  identify  the  limit  points  in  an  appropriate  manner. 

Stability  estimates  obtained  in  this  work  are  also  useful  for  the  study  of  the  uncontrolled  set¬ 
ting.  We  show  that,  under  appropriate  conditions,  stationary  distributions  of  scaled  queue  length 
processes  (which  are  strong  Markov  processes  when  there  are  no  controls)  converge  to  the  unique 
stationary  distribution  of  the  limit  reflecting  diffusion  model.  For  the  setting  where  the  arrival  and 
service  rates  are  constant,  such  a  result  has  been  proved  in  [13].  To  our  knowledge  the  current 
paper  is  the  first  one  to  treat  a  setting  with  state  dependent  rates. 
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C.III.  Large  Deviations  for  Stochastic  Flows  of  Diffeomorphisms  [11].  Stochastic  flows 
of  diffeomorphisms  have  been  a  subject  of  much  research.  In  this  paper,  we  are  interested  in  an 
important  subclass  of  such  flows,  namely  the  Brownian  flows  of  diffeomorphisms.  Our  goal  is  to 
study  small  noise  asymptotics,  specifically,  the  large  deviation  principle  (LDP)  for  such  flows. 

Elementary  examples  of  Brownian  flows  are  those  constructed  by  solving  finite  dimensional  Ito 
stochastic  differential  equations.  More  precisely,  suppose  6,/,:,  i  =  1  are  functions  from 

x  [0,T]  to  that  are  continuous  in  (x,t)  and  (k  +  l)-times  continuously  differentiable  (with 
uniformly  bounded  derivatives)  in  x.  Let  f3\, . . .  j3m  be  independent  standard  real  Brownian  motions 
on  some  filtered  probability  space  (Q,  T ,  P,  {Ft})-  Then  for  each  s  G  [0,  T]  and  x  G  Md,  there  is  a 
unique  continuous  {Ft}— adapted,  Wl— valued  process  0s,t(x),  s  <t  <T,  satisfying 

fit  171  fit 

<l>s,t(x)  =  x+  b(cj)Sjr(x),r)dr  +  ^  /  fi((j>s,r(x),r)d/3i(r).  (17) 

Js  i=1  Js 

By  choosing  a  suitable  modification,  {4>s,t,  0  <  s  <  t  <  T}  defines  a  Brownian  flow  of  Cfc— diffeomorphisms. 
In  particular,  denoting  by  Gk  the  topological  group  of  Cfc-diffeomorphisms,  one  has  that  <p  = 
{^o,t)0  <  t  <  T}  is  a  random  variable  with  values  in  the  Polish  space  =  C'([0,  T]  :  Gk).  For 
e  G  (0,  oo),  when  /)  is  replaced  by  efi  in  (17),  we  write  the  corresponding  flow  as  <p£.  Large 
deviations  for  cj)£  in  ,  as  e  — *  0,  have  been  studied  in  several  works. 

As  is  well  known,  not  all  Brownian  flows  can  be  expressed  as  in  (17)  and  in  general  one  needs 
infinitely  many  Brownian  motions  to  obtain  a  stochastic  differential  equation  (SDE)  representation 
for  the  flow.  Indeed  typical  space-time  stochastic  models  with  a  realistic  correlation  structure  in  the 
spatial  parameter  naturally  lead  to  a  formulation  with  infinitely  many  Brownian  motions  (one  such 
example,  coming  from  problems  in  image  analysis,  is  studied  in  this  work).  Thus,  following  Kunita’s 
notation  for  stochastic  integration  with  respect  to  semi-martingales  with  a  spatial  parameter,  the 
study  of  general  Brownian  flows  of  Cfc-diffeomorphisms  leads  to  SDEs  of  the  form 

d(j>s,t(x)  =  F(<j>Sjt(x),dt),  4>s,s(x )  =  x>  0  <  s  <t  <T,  x  G  Rd,  (18) 

where  F(x,t)  is  a  Ck+1  Brownian  motion.  Note  that  such  an  F  can  be  regarded  as  a  random 
variable  with  values  in  the  Polish  space  Wk  =  C([0,  T]  :  Cfc+1(Md)),  where  Cfc+1(Md)  is  the  space  of 
(fc+1)  times  continuously  differentiable  functions  from  IRd  to  Md.  Representations  of  such  Brownian 
motions  in  terms  of  infinitely  many  independent  standard  real  Brownian  motions  is  well  known. 
Indeed,  one  can  represent  F  as 

fit  °°  fit 

F(x,t)=  /  b(x,r)dr  +  V]  /  fi(x,r)d(3i(r)1  (x,  t)  G  Rd  x  [0,  T],  (19) 

Jo  i=1  Jo 

where  is  an  infinite  sequence  of  i.i.d.  real  Brownian  motions  and  b,  ft  are  suitable  functions 

from  x  [0,  T]  to  . 

Letting  a(x,y,t)  =  fi(x^)f{{y,t)  for  x,y  G  Md,  t  G  [0,T],  the  functions  (a,  b)  are  referred 

to  as  the  local  characteristics  of  the  Brownian  motion  F.  When  equation  (18)  is  driven  by  the 
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Brownian  motion  Fe  with  local  characteristics  ( sa,b ),  we  will  denote  the  corresponding  solution 
by  (f)£.  In  this  work  we  establish  a  large  deviation  principle  for  (cj)£,F£)  in  W^-i  x  W^-i-  Note 
that  the  LDP  is  established  in  a  larger  space  than  the  one  in  which  {<jf,F£)  take  values  (namely, 
Wk  x  Wk).  This  is  consistent  with  results  in  literature,  which  consider  stochastic  flows  driven  by 
only  finitely  many  real  Brownian  motions.  The  main  technical  difficulty  in  establishing  the  LDP 
in  Wk  x  Wk  is  the  proof  of  tightness  of  certain  controlled  processes,  when  fc  —  1  is  replaced  by  k. 

As  noted  above,  the  stochastic  dynamical  systems  considered  in  this  work  are  driven  by  an 
infinite  dimensional  Brownian  motion.  A  broadly  applicable  approach  to  the  study  of  large  devia¬ 
tions  for  such  systems,  based  on  variational  representations  for  functionals  of  infinite  dimensional 
Brownian  motions,  has  been  developed  in  [9,  10].  Several  authors  have  adopted  this  approach  to  an¬ 
alyze  the  large  deviation  properties  of  a  variety  of  models,  including  stochastic  PDEs  with  random 
dynamic  boundary  conditions,  stochastic  Navier-Stokes  equations  and  infinite  dimensional  SDEs 
with  non-Lipschitz  coefficients.  The  approach  is  a  particularly  attractive  alternative  to  standard 
discretization/approximation  methods  when  the  state  spaces  are  non-standard  function  spaces, 
such  as  the  space  of  diffeomorphisms  used  in  the  present  paper. 

The  proof  of  our  main  result  proceeds  by  verification  of  a  general  sufficient  condition  obtained  in 
[10]  (see  Assumption  2  and  Theorem  6  therein).  The  verification  of  this  condition  essentially  trans¬ 
lates  into  establishing  weak  convergence  of  certain  stochastic  flows  defined  via  controlled  analogues 
of  the  original  model.  These  weak  convergence  proofs  proceed  by  first  establishing  convergence  for 
N— point  motions  of  the  flow  and  then  using  Sobolev  and  Rellich-Kondrachov  embedding  theorems 
to  argue  tightness  and  convergence  as  flows.  The  key  point  here  is  that  the  estimates  needed  in 
the  proofs  are  precisely  those  that  have  been  developed  in  literature  for  general  qualitative  analysis 
(e.g.  existence,  uniqueness)  of  the  uncontrolled  versions  of  the  flows.  Unlike  past  approaches  taken 
for  the  study  of  (finite  dimensional)  stochastic  flows,  the  proof  of  the  LDP  does  not  require  any 
exponential  probability  estimates  or  discretization/approximation  of  the  original  model. 

We  next  study  an  application  of  these  results  to  a  problem  in  image  analysis.  Stochastic  dif- 
feomorphic  flows  have  been  suggested  for  modeling  prior  statistical  distributions  on  the  space  of 
possible  images/targets  of  interest  in  the  study  of  nonlinear  inverse  problems  in  this  field.  Along 
with  a  data  model,  noise  corrupted  observations  with  such  a  prior  distribution  can  then  be  used  to 
compute  a  posterior  distribution  on  this  space,  the  “mode”  of  which  yields  an  estimate  of  the  true 
image  underlying  the  observations.  Motivated  by  such  a  Bayesian  procedure  a  variational  approach 
to  this  image  matching  problem  has  been  suggested  and  analyzed  in  [22] .  In  this  work  we  develop  a 
rigorous  asymptotic  theory  that  relates  standard  stochastic  Bayesian  formulations  of  this  problem, 
in  the  small  noise  limit,  with  the  deterministic  variational  approach  taken  in  [22]. 


C.IV.  Feller  and  Stability  Properties  of  the  Nonlinear  Filter  [8],  The  classical  model 
of  nonlinear  filtering  consists  of  a  pair  of  stochastic  processes  (Xt,Yt)t>o  where  (Xt)  is  called  the 
signal  process  and  (Yt)  the  observation  process.  The  signal  is  taken  to  be  a  Markov  process  with 
values  in  some  Polish  space  S  and  the  observations  are  given  via  the  relation: 

Yt  =  f  h(Xs)ds  +  Wt,  (20) 

Jo 
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where  {Wtj  is  a  standard  d  dimensional  Brownian  motion  independent  of  (Xt)  and  h,  referred  to  as 
the  observation  function,  is  a  map  from  S  — >  Mrf.  Nonlinear  filtering  concerns  with  the  study  of  the 
measure  valued  process  (lit)  which  is  the  conditional  distribution  of  Xt  given  34  =  &{YS  :  0  <  s  <  t}. 
This  measure  valued  process  is  called  the  nonlinear  filter.  In  a  very  influential  paper,  Kunita[24]  has 
shown,  using  uniqueness  of  solutions  to  Kushner-Stratonovich  equations,  that  in  the  above  filtering 
model  if  the  signal  is  Feller-Markov  with  a  compact  Hausdorff  state  space  S  then  the  nonlinear  filter 
is  also  a  Feller-Markov  process  with  state  space  V(S),  where  V(S)  is  the  space  of  all  probability 
measures  on  S  .  Furthermore,  [24]  provides  a  proof  of  the  statement  that  if  the  signal  has  a  unique 
invariant  measure  fi  then,  under  a  minor  additional  condition,  the  filter  has  a  unique  invariant 
measure.  Recently,  Baxendale  et  al  [6]  have  found  a  gap  in  the  proof  of  this  fundamental  result.  In 
this  survey  article  we  revisit  this  work  of  Kunita  and  several  other  works  that  extend  or  make  use 
of  [24] ,  in  light  of  the  discovery  made  in  [6] . 

We  first  consider  the  Feller  property  and  existence  of  invariant  measures.  It  is  shown  that  these 
properties  hold  in  a  much  greater  generality  than  the  one  originally  considered  in  [24] .  We  then  turn 
to  the  study  of  ergodicity  of  the  nonlinear  filter.  We  recall  the  gap  pointed  out  by  Baxendale  et  al 
in  [6].  The  main  issue  is  the  equality  of  certain  a— fields  claimed  in  [24].  To  illustrate  the  difficulty, 
consider  a  probability  space  (Ho,  Fo,  Po)  with  sub-  a-  fields  T* ,  (G*)n> o,  such  that  all  the  a  fields 
are  Po  complete  and  {Gn}n> o  is  a  decreasing  family  of  a  fields  such  that  fjn>0  @ n  is  -fb  trivial. 
Then  the  equality  claimed  in  [24]  is  analogous  to  the  statement  that  7i*  =  fjn>o  V  G*x)  =  X* . 
This  equality  holds  in  many  circumstances  -  for  instance  when  T*  is  independent  of  Gq  -  however, 
it  is  false  in  general.  Clearly  TL*  D  T* .  but  one  can  construct  examples  that  show  that  the  reverse 
inclusion  can  fail.  In  this  work  we  give  some  sufficient  conditions  under  which  the  equality  of  the 
a—  fields  stated  in  [24]  is  true.  These  sufficient  conditions  are  phrased  in  terms  of  certain  stability 
properties  of  the  nonlinear  filter,  eg.  finite  memory  property  or  asymptotic  stability.  Previous 
works  by  several  authors  on  asymptotic  stability  of  the  filter  for  a  variety  of  filtering  models,  then 
identify  a  rich  class  of  filtering  problems  for  which  the  uniqueness  of  invariant  measures  for  the 
nonlinear  filter  holds.  We  in  fact  find  that  the  equality  of  a—  fields  stated  in  [24]  is  equivalent  to 
several  asymptotic  properties  of  the  nonlinear  filter,  such  as,  uniqueness  of  invariant  measure  for 
the  nonlinear  filter,  uniqueness  of  invariant  measure  for  the  signal-filter  pair,  asymptotic  stability 
and  finite  memory  of  the  filter. 


C.V.  Modified  particle  filter  methods  for  assimilating  Lagrangian  data  into  a  point- 
vortex  model  [28], 

The  issues  surrounding  the  assimilation  of  data  into  models  have  recently  gained  prominence  in 
almost  all  areas  of  geophysics.  The  quantity  and  quality  of  data  has  increased  dramatically  with 
ever-improving  observational  technologies.  At  the  same  time,  our  ability  to  run  extensive  simula¬ 
tions  on  ever-faster  computers  has  enhanced  the  use  of  models.  Data  assimilation  is  the  problem  of 
estimating  the  optimal  prediction  that  combines  model  output,  with  its  attendant  uncertainty,  and 
observations,  which  will  also  contain  errors.  Most  of  the  traditional  techniques  of  data  assimilation 
are  based  on  (linear)  control  theory  and  optimization.  Their  adaptation  to  highly  nonlinear  fluid 
flows,  as  in  atmospheric  and  oceanic  dynamics,  presents  many  mathematical  challenges.  We  focus 
here  on  one  particular  context  in  which  such  nonlinear  behavior  occurs,  namely  the  trajectory 
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(Lagrangian)  motion  in  point  vortex  flow. We  take  an  approach,  based  on  the  statistical  method 
of  particle  filtering,  which  is  not  based  on  any  linearization  and  hence  accounts  well  for  nonlinear 
effects.  Lagrangian  observations  refer  to  sequences  of  position  measurements  along  trajectories  of 
(Lagrangian)  tracers.  Such  data  are  abundant  in  geophysical  systems.  In  contrast  to  the  assimi¬ 
lation  of  Eulerian  data,  assimilation  of  Lagrangian  data  poses  several  complications.  The  reason 
is  that  most  numerical  models  for  geophysical  systems  are  solved  on  a  fixed  grid  in  space  or  as 
spectral  model  and  do  not  relate  to  the  Lagrangian  observations  directly  in  terms  of  the  model 
variables.  Thus  the  conventional  approach  for  assimilating  Lagrangian  data  is  to  transform  them 
into  the  Eulerian  velocity  data.  A  new  approach  for  assimilating  Lagrangian  data  into  the  fluid  dy¬ 
namical  systems,  the  so-called  Langrangian  data  assimilation  (LaDA)  method,  was  introduced  by 
Ide  et  al [23] .  Although  the  LaDA  method  has  been  shown  to  be  more  efficient  than  conventional 
approaches,  its  formulation,  which  is  based  on  either  the  EKF  or  the  Ensemble  Kalman  Filter, 
approximates  the  model  uncertainty  only  up  to  second  order  since  it  uses  linearization  schemes. 
When  model  nonlinearity  is  strong  and  the  observation  period  is  long,  this  approximation  may 
become  invalid  and  filter  divergence  can  occur  The  particle  filter  (PF),  in  contrast,  is  an  approach 
in  which  a  full  distribution  of  the  model  uncertainty  is  attained  by  a  large  number  of  particles.  In 
other  words,  the  resulting  distributions  are  not  approximated  by  Gaussians  in  the  process  of  par¬ 
ticle  filtering.  Highly  nonlinear  effects  in  the  flow  can  therefore  be  accommodated  without  causing 
breakdown  of  the  filter.  In  this  paper,  we  implement  the  PF  with  the  LaDA  into  the  point  vor¬ 
tex  model  and  investigate  the  filter  performance.  Owing  to  small  dimensionality  and  yet  complex 
nonlinear  (Lagrangian)  dynamics,  vortex  models  are  a  natural  paradigm  for  the  investigation  of 
data  assimilation  system  with  Lagrangian  or  Eulerian  observations  .  Several  improved  PFs  with 
LaDA  tailored  to  the  point  vortex  model  are  developed  and  the  PF  results  with  the  EKF  results 
are  compared.  We  introduce  the  idea  of  occasional  backtracking  for  the  PF  to  overcome  suspected 
divergence  and  describe  three  specific  reinitialization  schemes  implemented  with  backtracking.  We 
conduct  numerical  experiments  and  compare  the  performance  of  several  PFs  and  the  EKF  applied 
to  the  point-vortex  system.  We  find  that  the  backtracking  particle  filters  (BPF)  introduced  in  this 
paper  and  applied  to  the  two-point  vortex  system  are  an  effective  assimilation  tool  for  this  complex 
nonlinear  system.  The  cloud  expanding  BPF  and  the  directed  doubling  BPF  both  achieved  lowest 
failure  rates  (for  nearly  every  case)  in  the  experiment  testing  dependence  of  initial  condition.  In  the 
experiment  testing  dependence  on  observation  period,  all  particle  filter  methods  outperformed  the 
EKF.  Typically,  for  a  given  application,  any  data  assimilation  method  requires  some  application- 
specific  tuning  to  perform  well.  The  ideas  we  introduced  here  for  particle  filters  in  the  context 
of  the  point-vortex  model  -  specifically  backtracking  to  a  previous  observation  time,  reinitializing 
the  filter,  doubling  the  number  of  particles,  and  running  assimilation  forward  -  are  effective  at 
overcoming  filter  divergence,  and  general  enough  to  be  applied  to  other  nonlinear  physical  systems. 


C.VI.  Dissertation  abstract  of  Ph.D.  student  V.  Maroulas.  Large  deviations  theory  con¬ 
cerns  with  study  of  precise  asymptotics  governing  the  decay  rate  of  probabilities  of  rare  events.  A 
classical  area  of  large  deviations  is  the  FreidlinWentzell  (FW)  theory  that  deals  with  path  prob¬ 
ability  asymptotics  for  small  noise  Stochastic  Dynamical  Systems  (SDS).  For  finite  dimensional 
SDS,  FW  theory  has  been  very  well  studied.  The  goal  of  the  present  work  is  to  develop  a  system¬ 
atic  framework  for  the  study  of  FW  asymptotics  for  infinite  dimensional  SDS.  Our  first  result  is 
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a  general  LDP  for  a  broad  family  of  functionals  of  an  infinite  dimensional  small  noise  Brownian 
motion  (BM).  Depending  on  the  application,  the  driving  infinite  dimensional  BM  may  be  given 
as  a  spacetime  white  noise,  a  Hilbert  space  valued  BM  or  a  cylindrical  BM.  We  provide  sufficient 
conditions  for  LDP  to  hold  for  all  such  different  model  settings.  As  a  first  application  of  these 
results  we  study  FW  LDP  for  a  class  of  stochastic  reaction  diffusion  equations.  The  model  that 
we  consider  has  been  widely  studied  by  several  authors.  Two  main  assumptions  imposed  in  all 
previous  studies  are  the  boundedness  of  the  diffusion  coefficient  and  a  certain  geometric  condition 
on  the  underlying  domain.  These  restrictive  conditions  are  needed  in  proofs  of  certain  exponential 
probability  estimates  that  form  the  basis  of  classical  proofs  of  LDPs.  Our  proofs  instead  rely  on 
some  basic  qualitative  properties,  eg.  existence,  uniqueness,  tightness,  of  certain  controlled  ana¬ 
logues  of  the  original  systems.  As  a  result,  we  are  able  to  relax  the  two  restrictive  requirements 
described  above.  As  a  second  application  we  study  large  deviation  properties  of  certain  stochastic 
diffeonrorphic  flows  driven  by  an  infinite  sequence  of  i.i.d.  standard  real  BMs.  LDP  for  small  noise 
finite  dimensional  flows  has  been  studied  by  several  authors.  Typical  space  time  stochastic  models 
with  a  realistic  correlation  structure  in  the  spatial  parameter  naturally  leads  to  infinite  dimensional 
flows.  We  establish  a  LDP  for  such  flows  in  the  small  noise  limit.  We  also  apply  our  result  to  a 
Bayesian  formulation  of  an  image  analysis  problem.  An  approximate  maximum  likelihood  property 
is  shown  for  the  solution  of  an  optimal  image  matching  problem  that  involves  the  large  deviation 
rate  function. 


C.VII.  Dissertation  abstract  of  Ph.D.  student  C.  Lee.  Stochastic  processing  networks  arise 
commonly  from  applications  in  computers,  telecommunications,  and  large  manufacturing  systems. 
Study  of  stability  and  control  for  such  networks  is  an  active  and  important  area  of  research.  In 
general  the  networks  are  too  complex  for  direct  analysis  and  therefore  one  seeks  tractable  approxi¬ 
mate  models.  Heavy  traffic  limit  theory  yields  one  of  the  most  useful  collection  of  such  approximate 
models.  Typical  results  in  the  theory  say  that,  when  the  network  processing  resources  are  roughly 
balanced  with  the  system  load,  one  can  approximate  such  systems  by  suitable  diffusion  processes 
that  are  constrained  to  live  within  certain  polyhedral  domains  (e.g.,  positive  orthants).  Stability 
and  control  problems  for  such  diffusion  models  are  easier  to  analyze  and,  once  these  are  resolved, 
one  can  then  infer  stability  properties  and  construct  good  control  policies  for  the  original  physi¬ 
cal  networks.  In  my  dissertation  we  consider  three  related  problems  concerning  stability  and  long 
time  control  for  such  networks  and  their  diffusion  approximations.  In  the  first  part  of  the  disser¬ 
tation  we  establish  results  on  long  time  asymptotic  properties,  in  particular  geometric  ergodicity, 
for  limit  diffusion  models  obtained  from  heavy  traffic  analysis  of  stochastic  networks.  The  results 
address  the  rate  of  convergence  to  steady  state,  moment  estimates  for  steady  state,  uniform  in 
time  moment  estimates  for  the  process  and  central  limit  type  results  for  time  averages  of  such  pro¬ 
cesses.  In  the  second  part  of  the  dissertation  we  consider  invariant  distributions  of  an  important 
subclass  of  stochastic  networks,  namely  the  generalized  Jackson  networks  (GJN).  It  is  shown  that, 
under  natural  stability  and  heavy  traffic  conditions,  the  invariant  distributions  of  GJN  converge 
to  unique  invariant  probability  distribution  of  the  corresponding  constrained  diffusion  model.  The 
result  leads  to  natural  methodologies  for  approximation  and  simulation  of  steady  state  behavior  of 
such  networks.  In  the  final  part  of  the  dissertation  we  consider  a  rate  control  problem  for  stochas¬ 
tic  processing  networks  with  an  ergodic  cost  criterion.  It  is  shown  that  value  functions  and  near 
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optimal  controls  for  limit  diffusion  models  serve  as  good  approximations  for  the  same  quantities 
for  certain  physical  networks  that  are  heavily  loaded. 
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